Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering

Recent developments in pre-trained neural language modeling have led to leaps accuracy on common-sense question-answering benchmarks. However, there is increasing concern that models overfit specific tasks, without learning utilize external knowledge or perform general semantic reasoning. In contrast, zero-shot evaluations shown promise as a more robust measure of model’s reasoning abilities. this paper, we propose novel neuro-symbolic framework for question answering across commonsense tasks. Guided by set hypotheses, the studies how transform various pre-existing resources into form most effective pre-training models. We vary models, training regimes, sources, and data generation strategies, their impact Extending prior work, devise compare four constrained distractor-sampling strategies. provide empirical results five tasks with generated from resources. show that, while an individual graph better suited global brings consistent gains different addition, both preserving structure task well generating fair informative questions help learn effectively.